Landmark Detection for Unconstrained Face Recognition

In this dissertation a novel method for 3D landmark detection and pose estimation, suitable for both frontal and side 3D facial scans, is presented. It exploits 3D and 2D information by using local shape descriptors to extract candidate interest points that are subsequently identified and labeled as anatomical landmarks. Additionally, a novel generalized framework for combining facial feature descriptors that can be used for landmark detection is introduced, and several feature fusion schemes are proposed and evaluated. However, feature detection methods which use general purpose shape descriptors cannot identify and label the detected candidate landmarks. To this end, a 3D Facial Landmark Model (FLM) of facial anatomical landmarks is introduced. Candidate landmarks, irrespectively of the way they are generated, can be identified and labeled by matching them with the FLM. Finally, a novel method for unconstrained face recognition is introduced. It employs the 3D landmark detector to provide an initial pose estimation and to indicate occluded areas with missing data for each facial scan. Subsequently, a 3D Annotated Face Model (AFM) is registered and fitted to the scan using facial symmetry to complete the occluded areas. Using a biometric signature resulted from the wavelet representation of the fitted AFM, the proposed method can perform comparisons among interpose facial scans, unlike previously proposed methods that require frontal scans.